RESUMO
Results of simulation studies evaluating the performance of statistical methods can have a major impact on the way empirical research is implemented. However, so far there is limited evidence of the replicability of simulation studies. Eight highly cited statistical simulation studies were selected, and their replicability was assessed by teams of replicators with formal training in quantitative methodology. The teams used information in the original publications to write simulation code with the aim of replicating the results. The primary outcome was to determine the feasibility of replicability based on reported information in the original publications and supplementary materials. Replicasility varied greatly: some original studies provided detailed information leading to almost perfect replication of results, whereas other studies did not provide enough information to implement any of the reported simulations. Factors facilitating replication included availability of code, detailed reporting or visualization of data-generating procedures and methods, and replicator expertise. Replicability of statistical simulation studies was mainly impeded by lack of information and sustainability of information sources. We encourage researchers publishing simulation studies to transparently report all relevant implementation details either in the research paper itself or in easily accessible supplementary material and to make their simulation code publicly available using permanent links.
RESUMO
Accurate prediction of response to neoadjuvant chemotherapy (NAC) can help tailor treatment to individual patients' needs. Little is known about the combination of liquid biopsies and computer extracted features from multiparametric magnetic resonance imaging (MRI) for the prediction of NAC response in breast cancer. Here, we report on a prospective study with the aim to explore the predictive potential of this combination in adjunct to standard clinical and pathological information before, during and after NAC. The study was performed in four Dutch hospitals. Patients without metastases treated with NAC underwent 3 T multiparametric MRI scans before, during and after NAC. Liquid biopsies were obtained before every chemotherapy cycle and before surgery. Prediction models were developed using penalized linear regression to forecast residual cancer burden after NAC and evaluated for pathologic complete response (pCR) using leave-one-out-cross-validation (LOOCV). Sixty-one patients were included. Twenty-three patients (38%) achieved pCR. Most prediction models yielded the highest estimated LOOCV area under the curve (AUC) at the post-treatment timepoint. A clinical-only model including tumor grade, nodal status and receptor subtype yielded an estimated LOOCV AUC for pCR of 0.76, which increased to 0.82 by incorporating post-treatment radiological MRI assessment (i.e., the "clinical-radiological" model). The estimated LOOCV AUC was 0.84 after incorporation of computer-extracted MRI features, and 0.85 when liquid biopsy information was added instead of the radiological MRI assessment. Adding liquid biopsy information to the clinical-radiological resulted in an estimated LOOCV AUC of 0.86. In conclusion, inclusion of liquid biopsy-derived markers in clinical-radiological prediction models may have potential to improve prediction of pCR after NAC in breast cancer.
